Limitations of security data
We are constantly battered by cybersecurity data, reports, and marketing collateral—and we shouldn’t treat all of this information equally. Security data has inherent limitations and biases, which result in varying value and relevance in how it should be applied. It is important to understand which data is significant and how best to allow it to influence your decisions.
A tsunami of security metrics, reports, analyses, blogs, papers, and articles vie for our attention. Sources range from reporters, researchers, professional security teams, consultants, dedicated marketing groups, and even security-operations people who are adding data, figures, and opinions to the cauldron. We are flooded with data and opinions.
It was not always this way. More than a decade ago, we lived in an information desert, where even speculations were rare. Making decisions driven by data has always been a good practice. Years ago, many advocates were working hard to convince the industry to share information. Even a drop is better than none. Most groups that were capturing metrics were too frightened or embarrassed to share. Data was kept secret by everyone while decision makers were clamoring for security insights based upon industry numbers, which simply were not available.
The impact of data secrecy
In the past, fear, uncertainty, and doubt ruled. People expected the worst and unscrupulous security marketing advocates took advantage, fanning the flames to sell products and snake oil. Those were dark times, promulgated with outlandish claims of easily eradicating cyber threats with their software or appliance products. The market was riddled with magic boxes, silver-bullet software, and turnkey solutions to easily fix all security woes. I can remember countless salespeople asserting “we solve security” (at which point I stopped listening or kicked them out). The concept of flipping a switch to make all the complex problems of computing security forever go away was what uninformed organizations wanted to hear, but that was simply unrealistic. Why customers chose to believe such nonsense (when the problem and the effectiveness of potential solutions could not be quantified) is beyond me, but many did. Trust in the security solutions industry was lost for a time.
Slowly, a trickle of informative sources began to produce reports and publish data. Such initiatives gained momentum, with others joining to share in limited amounts. This was the turning point. Armed with data and critical thinking, clarity and common sense began to take root. The transition was not perfect or quick, but the introduction of data from credible sources empowered security organizations to better understand the challenge and effectively maneuver against threats.
As the size of the market and competition grew, additional viewpoints joined the fray. Today, we are bombarded by all manner of cybersecurity information. Some are credible, while others are not. There are several types of data being presented, ranging from speculations to hard research. Being well informed is extremely valuable to decision makers. Now, the problem is figuring out how to filter and organize the data so that we are not misled.
As part of my role as a cybersecurity strategist, I both publish information to the community and consume vast amounts of industry data. To manage the burden and avoid the risks of believing less-than-trustworthy information, I have created a quick guide to help structure the process. It is burned into my mind as a set of filters and rules, and I have committed it to paper/screen to share.
I categorize data into four buckets. These are speculation, survey, actuarial, and research. Each has its pros and cons. The key to managing security data overload is to understand the limitations of each category, its respective value and its recommended usage.
For example, survey data is the most unreliable, but does have value in helping us understand the fears and perceptions of the respondent community. Research data is normally very accurate but notoriously narrow in scope and may be late to the game. One of my favorites is actuarial data. I am a pragmatic guy. I want to know what is actually happening so I can make my own conclusions. But there are limitations to actuarial data as well. It tends to be very limited in size and scope, so you can’t look too far into it. It is a reflection of the past, which may not align to the future.
I hear lots of complaints and criticisms regarding the validity, scope, intent, and usage of data. I have my favorites and those that I refuse to even read. Security data is notoriously difficult. There are so many limitations and biases, it is far easier to point out problems than to see the diamonds in the rough. But data can be valuable if filtered, corrected for bias, and the limitations are known. Don’t go in blind. Apply common sense. Follow a consistent method and structure to avoid pitfalls and maximize the data available to help you manage and maintain an optimal level of security.
The following are a few examples, in my opinion, of credible cybersecurity data across the spectrum of categories. Keep in mind the limitations of each group and don’t make the mistake of using the information improperly! Look to speculation for the best opinions, survey for the pulse of industry perceptions, actuarial for real events, and research for deep analysis:
Speculation
- 2016 Cybersecurity Threat Predictions from McAfee Labs.
- $243 billion–$1 trillion: The potential cost of a single attack against the US power grid, per Lloyds Insurance.
- ~$3 trillion: The aggregate economic impact of cybersecurity on technology trends through 2020, from the World Economic Forum 2014 report “Risk and Responsibility in a Hyperconnected World.”
- $90 trillion: The cyber impact for one (worst case) scenario affecting the global benefits of information and communications technologies by 2030, from the Atlantic Council’s report estimate.
- 55% compound annual growth rate: The growth of the global Internet of Things security market for the period 2016–2020, from ResearchandMarkets.com.
- My 2016 Cybersecurity Predictions: Most of my blogs are in the speculation category.
Survey
- Threat Intelligence Sharing survey: McAfee Labs Threats Report March 2016.
- 20% jump in cybercrime in the United Kingdom since 2014, with nearly two-thirds of businesses expressing no confidence in the ability of law enforcement to deal with it, from PwC.
- 25% Americans believe they have experienced a data breach or cyber attack, from a Travelers survey.
- 43% of organizations surveyed indicated increases in cybersecurity will drive the most technology spending, from the 2016 ESG IT spending intentions research report.
- 61% of CEOs believe cyber threats pose a danger to corporate growth, from a PwC survey.
Actuarial
- 3 out of 5 Californians were victims of data breaches in 2015, according to the state attorney general, from the 2016 California Data Breach Report.
- ~35% of the US population: Top 10 health care breaches of 2015 affected about one-third of the US population, from the Department of Health and Human Services Office for Civil Rights.
- Data Breach Investigations Report, from Verizon.
- 2016 Annual Security Report, from Cisco.
- 42 million new unique pieces of malware discovered in Q4 2015, bringing the total known samples to almost 500 million, from the McAfee Labs Threats Report of March 2016.
- Security Intelligence Report, from the biannual report by Microsoft.
Research
- $325 million: Losses attributed to CryptoWall v3 ransomware, from analysis by the Cyber Threat Alliance.
- $13.1 billion: US government spending on cybersecurity in 2015, from FISMA report from the OMB.
- “Carbanak”: Advanced attack analysis from Kaspersky Lab.
By the way, this very blog should be considered speculation. Treat it as such.
Interested in more? Follow me on Twitter (@Matt_Rosenquist) and LinkedIn to hear insights and what is going on in cybersecurity.
Stay Updated
Follow us to stay updated on all things McAfee and on top of the latest consumer and mobile security threats.
